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DETAILED ACTION 

NOTE: This action has been made non-final due to Claims 19, 20, and 22 not falling 
within one of the four statutory categories of invention. Please see new 35 USC 1 01 
rejection below. 

Response to Arguments 

1 . Applicant's arguments filed 04/22/2010 have been fully considered but they are 

not persuasive. 

Argument (page 11 If 1 and 2): 

• "Alpdemir does not teach or suggest a plurality of semantic items being 
stored in an ordered list where additional information provided by a user in 
a response is associated with one or more semantic items in the ordered 
list and where the one or more semantic items are identified to depart from 
the selected order and generate an additional prompt, as recited in 
independent claim 1 1 . For instance, if a user says "more" the system of 
Alpdemir simply provides other business information results, if user says 
"help", the system of Alpdemir simply connects the user to automated 
help. In both cases, "more" and "help" are not additional information that 
are associated with semantic items in an ordered list as recited in claim 
11" 

Response to argument: 
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Re "a plurality of semantic items being stored in an ordered list where additional 
information provided by a user in a response is associated with one or more semantic 
items in the ordered list and where the one or more semantic items are identified to 
depart from the selected order and generate an additional prompt", Alpdemir teaches a 
call flow example, where both the content and ordering of the content may change to 
suit local or region speech patterns, the goals of the call service, the demographics of 
the caller to the extent they can be determined, and numerous other objective and 
subjective or cultural factors ([0251]). 

Additionally, Alpdemir teaches a second example of a call flow, where those having 
ordinary skill in the art in light of the description provided here will appreciate that the 
procedures described for existing registered users as well as for new business users 
may be modified to provide somewhat different options at each stage of the interaction 
or to provide different ordering of the options . Therefore, the interaction described here 
are merely exemplary of the type of business user to system interaction desirable in an 
implemented system, but does not limit the inventive system or method to these 
particular interaction schemes or procedures ([0221]). 

Further, Alpdemir teaches a category sponsorship message is a message will come up 
when the caller requests businesses in a category without a specific business name in 
mind. Then the system will play back the message of the sponsors in that category in a 
pre-determined order, random order or a dynamically defined order based on some 
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policy or membership level or category, or other rules . If the business user declines the 
opportunity to subscribe, the business user is thanked for providing the information 
(Step 334) and the interaction terminates (Step 336) . If the business user indicates a 
desire to subscribe, he or she is given opportunity to record a category sponsorship 
message (Step 328) and further opportunities to either approve or change the message 
until he or she is satisfied with the recorded category sponsor message (Step 330). The 
category sponsor message is then published (Step 332) and the business user is 
thanked for providing the information (Step 334) and the interaction terminates (Step 
336) ([0217]). 

Therefore, the combined teaching of Alpdemir in view of Albayrak and Bangalore would 
render obvious a plurality of semantic items being stored in an ordered list where 
additional information provided by a user in a response is associated with one or more 
semantic items in the ordered list and where the one or more semantic items are 
identified to depart from the selected order and generate an additional prompt and 
would allow for a voice enabled interface implementing the dynamic ordering of 
Alpdemir improved by the use of synonyms to handle a broad order (Bangalore [0057- 
0058] e.g. 1600 Pennsylvania Avenue (the White House) to 50 Massachusetts Ave 
(Union Station, Washington DC)), wherein it is well known that a user may not follow the 
order prompted by the system. 

Examiner maintains previous office action arguments as well. 



Application/Control Number: 10/755,623 
Art Unit: 2626 



Page 5 



Re Remarks (pages 10-1 1 with respect to claim 1), Examiner believes that while giving 
claims their broadest reasonable interpretation in light of the supporting disclosure 
without importing limitations from the specification into the claims unnecessarily, 
Alpdemir teaches the limitations of claim 1 . Consider the interpretation of "the set of 
controls includes attributes that define a selected order for execution of the plurality of 
question controls to generate the markup for an audible prompting of the plurality of 
questions in the selected order" in light of Fig. 16 of the present invention's Drawings. 
Alpdemir teaches various call flow scenarios that are subject to change depending on 
related responses from a user. Examiner believes that an order list or suggestion that 
an additional prompt can be provided, which departs from a selected order is necessary 
as a means to track the conversation of a user and computer. 

Alpdemir states that both the content and ordering of the content may change to suit 
local or region speech patterns, the goals of the call service, the demographics of the 
caller to the extent they can be determined, and numerous other objective and 
subjective or cultural factors ([0251]). Examiner believes this to be descriptive of the 
importance of the order in which questions and answers are presented. 

Consider the following teachings of Alpdemir having two examples, where explicit 
deviations exist that change additional prompts presented to a user, The first prompt 
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presented to the user immediately can diverge a conversation one of two ways as 
shown below. 

In this embodiment, a new business manager/owner calls in and registers to be a 
member. She/he is asked to provide a credit card number for business information 
verification and services. Then, she/he gets prompted to verify the existing information 
and subscribe 3 months free of charge to the basic services where she is told she can 
record her promotional or informational message at the prompt. Then she/he is given a 
chance to review the existing business information in the Talk41 1 database, edit 
information if not correct, review and publish. Then she is given a chance to review and 
publish her promotional or informational message and asked if she wants to learn about 
enhanced membership services. If she/he says yes, then she is briefly explained what 
those are and given a chance to subscribe. She/he is told she can always change her 
message, subscribe for additional services either on the telephone or on the 
Talk41 1 .com web site. 

First Exemplar Script for Consumer Call: 

"Hello, welcome to Talk41 1 , Bay Area's fresh information source" 
"Please speak clearly. You can cut me off at any time, I won't get offended" 
"If you are a member or want to become one, say "Yes\ if you want me to 
Find a business for you, say the name, or the type of the business" 
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"ABC Pizza is at 408-255-0000, say "connect", or say "more" for more info" 
More 

"There is no additional info. Do you want me to connect now or would you like me to 
recommend others in the area?" 

Others 

"Choice 1 : Domino. Choice 2: Straw Hat with 1 0% off on any large Pizza, say your 
choice" 

(CONDITIONAL STATEMENTS) 

If answer is: Straw Hat 

"You will need the following number to get your discount. Write it down 
while I connect. It is: SH1 . Again, it is SH1 . Thank you for calling us" 



If answer is: Domino's 
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"All of Domino's Pizzas are made with real cheese and guaranteed to be hot on 
delivery! Thank you for calling us." 



Exemplar Script for Business Call: 

"Hello, welcome to Talk41 1 , fresh info source for the Bay Area" 
"Please speak clearly." 

"If you are a member or want to become one say "Yes", if you want me to find a 
business for you, say the name or the type of business" 

I want to become a Member 

"What is your business name or business phone number?" 
ABC Pizza 

"I do have some info for your business. Please answer my questions so I can process 
your membership. Your first three months are free of charge. We do require a major 
credit card so we can continue to provide you with uninterrupted service and give you 
the ability to order audio promotions and coupons any time any where on the phone or 
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at the Talk41 1 web site. We will not charge your card until your 3-month trial period is 
complete. Please provide a MasterCard or a VISA credit card number followed with the 
expiration date" 

xxxx xxxx xxxx xxxx and the expiration is 03/01 

"What is the full name on the card?" 
John Doe 

"What is the zip code for the billing address?" 
94102 

"Please wait a moment while we process your information" 
(may play an ad here) 

"Thank you. Your password is PassOOl . Please write it down. Again it is PassOOl . 
Now you can dictate a short message that I will deliver to callers when they request 
your business information. You can change this message anytime here or at Talk41 1 
web site by signing in with your password. Please dictate at the prompt or say Exit" 
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ABC Pizza is located right next to the Great Mall and we serve anywhere in the 
BayArea 

"This is how I understood your dictation: "ABC Pizza is located right next to the Great 
Mall and we serve anywhere in the Bay Area". If this is not correct please repeat the 
message. If it is correct, say so" (after two tries, go to the Talk41 1 web site to type in 
your message). 

Correct 

"Congratulations. Now all callers who request your business info will hear your 
message. I can promote your business in other ways as well such as Category 
Sponsorships and Audio Coupons. Please visit our web site at www.Talk41 1 .com and 
give me a call again. When you are ready to purchase additional services you can say 
"Promote" any time and I will step you through it. Thank ou for your business" ([0252]- 
[0287]). 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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Claims 19, 20, and 22 are rejected under 35 U.S.C. 101 because: 
Claims 19, 20, and 22 do not fall within one of the four statutory categories of 
invention. Supreme Court precedent 1 and recent Federal Circuit decisions 2 indicate 
that a statutory "process" under 35 U.S.C. 101 must (1) be tied to another statutory 
category (such as a particular apparatus), or (2) transform underlying subject matter 
(such as an article or material) to a different state or thing. While the instant claim(s) 
recite a series of steps or acts to be performed, the claim(s) neither transform 
underlying subject matter nor positively tie to another statutory category that 
accomplishes the claimed method steps, and therefore do not qualify as a statutory 
process. 

Claims 19, 20, and 22 recite purely mental steps and would not qualify as a 
statutory process. In order to qualify as a statutory process, the method claim should 
positively recite the other statutory class to which it is tied (i.e. apparatus, device, 
product, etc.). Claims 19, 20, and 22 are to a system comprising "a set of controls", "a 
server module" and "a client module". Applicant discloses "controls comprise elements 
of a dialog such as a question, answer, confirmation, command or statement" (abstract) 
and "modules include routines, programs, objects, components, data structures, etc. 
that perform particular tasks or implement particular abstract data types" (PG Pub 



1 Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk 
v. Benson, 409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 U.S. 780, 787-88 (1876). 

2 In re Bilski, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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110038). As such, the claim is to a combination of dialog elements and computer 
software perse, neither of which is within the statutory classes of inventions. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1, 5, 7, 11, 14, and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Alpdemir US 20020035474 A1 (hereinafter Alpdemir) in view of 
Albayrak et al. US 6662163 B1 (hereinafter Albayrak) and further in view of Bangalore 
et al. US 20050135571 A1 (hereinafter Bangalore). 

Re claim 1 , Alpdemir teaches a computer readable storage medium having 
instructions, which when executed on a computer generate client side markup ([0140]) 
for a client in a client/server system, the instructions comprising: 

a module, when executed on a process of a computer associated with the client, 
creates a dialog ([0139]), wherein the client side markup includes markup related to 
audible prompting of a plurality of questions ([0226]) and markup related to a grammar 
for recognition ([0169]) as a function of responses from the user ([0143-0144]), wherein 
the client side markup is adapted to prioritize prompting of the plurality of questions 
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([0337-0339]) and generate audible prompts for the plurality of questions in a selected 
order ([0222 & 02223] & Fig. 5 ordered set of commands/responses) 

wherein using the client side markup, the dialog follows a selected order of 
prompting of the plurality of questions and receives input from a user 

wherein the dialog departs from the selected order as a function of responses 
from the user ([0143-0144]), when at least one response includes an answer to the 
prompt that was given and additional information that is provided by the user in the 
response with the answer, and is not an answer to the prompt that was given, 

wherein an additional prompt is then provided to the user concerning the 
additional information before a next prompt in the selected order ([0132], help 
information) before returning to the selected order ([0250-0338] & Fig. 5, examples 
illustrating prompt and response in a dialog environment, that can start a users dialog 
over again) and providing the next prompt in the selected order. 

However, Alpdemir fails to teach a client side markup for a client in a client/server 
system 

Albayrak teaches an interactive voice response system includes a server and a 
set of mobile clients. The server and clients include RF transceivers for exchanging 
messages over an RF channel. Each mobile client includes a microphone, a speaker or 
headset, a processor and a voice browser. The voice browser interprets voice pages 
received from the server. Upon receiving a particular voice page from the server, the 
voice browser outputs via the speaker voice prompts specified by the voice page. A 
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speech recognition engine used by the voice browser converts voice responses from a 
user into a text response. The voice browser then performs an action based on the text 
response. The action taken may be to request a new voice page from the server, or to 
continue to interpret the current voice page. The server preferably includes an HTTP 
server module for receiving and responding to requests for voice pages from the mobile 
clients in accordance with a predefined protocol. The mobile clients each include a text- 
to-speech module for converting text in a voice page into voice prompts, and a digitized 
speech module for playing digitized voice data representing other voice prompts. The 
mobile clients also include a speech recognition module for recognizing words or data 
string within a user's voice responses in accordance with a user specific voice file 
received from the server (Albayrak Col. 3 lines 3-27). 

Further, Albayrak teaches that Hypertext refers to a collection of computer- 
readable text documents containing links, that is, location references. A browser utilizes 
the links to facilitate moving its attention between linked documents. A voice browser is 
similar to a graphical browser in that it is a program that processes hypertext and 
presents the hypertext content in a specified format. The voice browser used in the 
preferred embodiment of this invention receives and outputs all information in the form 
of sound rather than having graphical input and output. The particular type of hypertext 
used in the preferred embodiment is based on VoiceXML. VoiceXML was designed by 
the VoiceXML Forum to create audio dialogs that feature digitized audio and speech 
recognition and to facilitate web-based development and content delivery. (Albayrak 
Col. 4 lines 11-39). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir to incorporate a client side 
markup for a client in a client/server system as taught by Albayrak to allow for a server- 
client voice browsing system, wherein markup language is utilized to convert text to 
speech, particularly for a mobile client that can prompt a user wirelessly (Albayrak Col. 
3 lines 3-27). 

However, Alpdemir in view of Albayrak fails to teach obtaining a value for at least 
one field of a form 

Bangalore teaches for example On Mapquest.RTM., to get driving directions, a 
user would click on the driving directions button 214. Other buttons include a maps 
button 212 and a road trip planner 216. FIG. 2B illustrates the forms to fill out for driving 
directions. On this webpage 230, there is a starting address 218 and a destination 
address 220. The information is filled into field 222 for a starting address and the field 
224 for a destination address. The present invention involves generating the necessary 
information drawn from the VoiceTone.sup.SM dialog to submit a request from the 
VoiceTone.sup.SM application to populate the necessary fields, for example in 
Mapquest, to obtain information. The interaction with the webpage 230 is all performed 
by VoiceTone.sup.SM or a process associated with VoiceTone.sup.SM inasmuch as the 
user is on a telephone call (Bangalore [0030]). 

Further, Bangalore teaches [0057] FIG. 2C illustrates the response from the 
website when a user inputs a starting address 1600 Pennsylvania Avenue (the White 
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House) to 50 Massachusetts Ave (Union Station, Washington DC). Information includes 
the total distance 1 .62 miles, and total estimates time: 6 minutes. Detailed directions 
are provided 242 to the user and several options enable the user to receive the fastest 
route 252, the shortest route 244 or a route that avoids highways 246. The user can 
receive the directions via email 254 or the directions can be sent to a PCS phone 248. 
The form parser will analyze either off-line or dynamically each of the webpages 
illustrates in 2A, 2B and 2C to generate the appropriate prompts and input tags to 
receive via a voice dialog the information. As an example, after receiving via the voice 
dialog the "from" and "to" addresses, the service will receive the directions webpage 
shown in FIG. 2C. The form parser 106 will then identify the various pieces of 
information on the webpage 250 and dynamically adjust the dialog to ask the user 
questions such as: "Would you like to receive the directions via email or by phone?" If 
the response is by email, then the service receives the email address from the user or 
from another database or storage location and completes the process. The service may 
provide the directions (steps 1-5 on the webpage 250) via the voice dialog (Bangalore 
[0057-0058]). 

Additionally, Bangalore implements communication between a human and a 
machine to fill fields of a form, wherein Bangalore teaches the improvement of user 
information by implementing various prompts rather than assuming a user does not 
need any additional information ([0038-0052], particularly [0045-0050] diverging a bit 
from the topic of conversation to assist the user). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak to incorporate 
obtaining a value for at least one field of a form as taught by Bangalore to allow for 
dynamically adjusting the dialog between a human and a computer (Bangalore [0057- 
0058]), wherein rather than assuming a user does not need any additional information, 
the system diverges from the topic of conversation to assist the user ([0038-0052], 
particularly [0045-0050], inquiring about a phone number to a location when a user did 
not inquire about a phone number to further assist before returning back to "is there 
anything else I can do for you today?"). 

Re claim 5, Alpdemir teaches the computer readable storage medium of claim 1 
wherein one of the controls provides means for defining a confirmation for generating 
markup related to confirming that a recognized result is correct ([0191]). 

Re claims 7 and 14, Alpdemir teaches the computer readable storage medium of 
claim 1 wherein the module maintains information related to an order of responses 
received from the user, and wherein the module departs from the selected order ([0337- 
0339]) to provide a prompt related to a previous [RLI0] response from the user in the 
information ([0143-0144]). 

However, Alpdemir in view of Albayrak fails to teach providing a prompt related to 
a previous [RLI0] response from the user. 
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Bangalore teaches for example On Mapquest.RTM., to get driving directions, a 
user would click on the driving directions button 214. Other buttons include a maps 
button 212 and a road trip planner 216. FIG. 2B illustrates the forms to fill out for driving 
directions. On this webpage 230, there is a starting address 218 and a destination 
address 220. The information is filled into field 222 for a starting address and the field 
224 for a destination address. The present invention involves generating the necessary 
information drawn from the VoiceTone.sup.SM dialog to submit a request from the 
VoiceTone.sup.SM application to populate the necessary fields, for example in 
Mapquest, to obtain information. The interaction with the webpage 230 is all performed 
by VoiceTone.sup.SM or a process associated with VoiceTone.sup.SM inasmuch as the 
user is on a telephone call (Bangalore [0030]). 

Further, Bangalore teaches [0057] FIG. 2C illustrates the response from the 
website when a user inputs a starting address 1600 Pennsylvania Avenue (the White 
House) to 50 Massachusetts Ave (Union Station, Washington DC). Information includes 
the total distance 1 .62 miles, and total estimates time: 6 minutes. Detailed directions 
are provided 242 to the user and several options enable the user to receive the fastest 
route 252, the shortest route 244 or a route that avoids highways 246. The user can 
receive the directions via email 254 or the directions can be sent to a PCS phone 248. 
The form parser will analyze either off-line or dynamically each of the webpages 
illustrates in 2A, 2B and 2C to generate the appropriate prompts and input tags to 
receive via a voice dialog the information. As an example, after receiving via the voice 
dialog the "from" and "to" addresses, the service will receive the directions webpage 
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shown in FIG. 2C. The form parser 106 will then identify the various pieces of 
information on the webpage 250 and dynamically adjust the dialog to ask the user 
questions such as: "Would you like to receive the directions via email or by phone?" If 
the response is by email, then the service receives the email address from the user or 
from another database or storage location and completes the process. The service may 
provide the directions (steps 1-5 on the webpage 250) via the voice dialog (Bangalore 
[0057-0058]). 

Additionally, Bangalore implements communication between a human and a 
machine to fill fields of a form, wherein Bangalore teaches the improvement of user 
information by implementing various prompts rather than assuming a user does not 
need any additional information ([0038-0052], particularly [0045-0050] diverging a bit 
from the topic of conversation to assist the user). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak to incorporate 
providing a prompt related to a previous [RLI0] response from the user as taught by 
Bangalore to allow for dynamically adjusting the dialog between a human and a 
computer (Bangalore [0057-0058], and Fig. 2A, voice to text input into fields on a form), 
wherein rather than assuming a user does not need any additional information, the 
system diverges from the topic of conversation to assist the user ([0038-0052], 
particularly [0045-0050], inquiring about a phone number to a location when a user did 
not inquire about a phone number to further assist before returning back to "is there 
anything else I can do for you today?"). 
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Re claim 1 1 , Alpdemir teaches a computer implemented method for performing 
recognition and/or audible prompting on a client in a client/server system, the method 
comprising: 

receiving client side markup, the client die markup including markup related to 
audible prompts of questions and markup related to a grammar used for recognition as 
a function of responses from a user ([0139]), wherein the markup defines selected order 
of the questions ([0226])to prioritize prompting of the questions in a dialog with a user 
([0337-0339]) 

creating the dialog on a client as a function of execution of the client side markup 
related to the controls using a processor of a computer, wherein the dialog follows a 
selected order of the question controls ([0217]) to obtain values from a user for fields of 
the form and includes 

storing a plurality of semantic items that maintain information related to 
responses received from the user for questions, the plurality of semantic items being 
stored in an ordered list ([0217]), 

a first prompt for a first question control in the selected order ([0309]), the first 
question control being associated with a first field of the form; 

before proceeding with a next question in the selected order, departing from the 
selected order of the questions by accessing the ordered list ([0337-0339]) of semantic 
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items and identifying the one or more semantic items associated with the additional 
information ([0217]) and, in response 

However, Alpdemir fails to teach a client side markup for a client in a client/server 
system 

Albayrak teaches an interactive voice response system includes a server and a 
set of mobile clients. The server and clients include RF transceivers for exchanging 
messages over an RF channel. Each mobile client includes a microphone, a speaker or 
headset, a processor and a voice browser. The voice browser interprets voice pages 
received from the server. Upon receiving a particular voice page from the server, the 
voice browser outputs via the speaker voice prompts specified by the voice page. A 
speech recognition engine used by the voice browser converts voice responses from a 
user into a text response. The voice browser then performs an action based on the text 
response. The action taken may be to request a new voice page from the server, or to 
continue to interpret the current voice page. The server preferably includes an HTTP 
server module for receiving and responding to requests for voice pages from the mobile 
clients in accordance with a predefined protocol. The mobile clients each include a text- 
to-speech module for converting text in a voice page into voice prompts, and a digitized 
speech module for playing digitized voice data representing other voice prompts. The 
mobile clients also include a speech recognition module for recognizing words or data 
string within a user's voice responses in accordance with a user specific voice file 
received from the server (Albayrak Col. 3 lines 3-27). 
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Further, Albayrak teaches that Hypertext refers to a collection of computer- 
readable text documents containing links, that is, location references. A browser utilizes 
the links to facilitate moving its attention between linked documents. A voice browser is 
similar to a graphical browser in that it is a program that processes hypertext and 
presents the hypertext content in a specified format. The voice browser used in the 
preferred embodiment of this invention receives and outputs all information in the form 
of sound rather than having graphical input and output. The particular type of hypertext 
used in the preferred embodiment is based on VoiceXML. VoiceXML was designed by 
the VoiceXML Forum to create audio dialogs that feature digitized audio and speech 
recognition and to facilitate web-based development and content delivery. (Albayrak 
Col. 4 lines 11-39). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir to incorporate a client side 
markup for a client in a client/server system as taught by Albayrak to allow for a server- 
client voice browsing system, wherein markup language is utilized to convert text to 
speech, particularly for a mobile client that can prompt a user wirelessly (Albayrak Col. 
3 lines 3-27). 

However, Alpdemir in view of Albayrak fails to teach obtaining a value for at least 
one field of a form 

receiving a user response toaudible prompt that includes an answer to the first 
question that is indicative of a value for the first field of the form, 
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an additional prompt for a question control associated with a second field of the 
form based on the one or more semantic items associated with the additional 
information, wherein providing the additional prompt departs from the selected order of 
the question controls ([0217]); and 

after the user has provided an answer to the additional prompt, returning to the 
selected order of the question controls to provide a next prompt for a next question 
control in the selected order ([0217]). 

Bangalore teaches for example On Mapquest.RTM., to get driving directions, a 
user would click on the driving directions button 214. Other buttons include a maps 
button 212 and a road trip planner 216. FIG. 2B illustrates the forms to fill out for driving 
directions. On this webpage 230, there is a starting address 218 and a destination 
address 220. The information is filled into field 222 for a starting address and the field 
224 for a destination address. The present invention involves generating the necessary 
information drawn from the VoiceTone.sup.SM dialog to submit a request from the 
VoiceTone.sup.SM application to populate the necessary fields, for example in 
Mapquest, to obtain information. The interaction with the webpage 230 is all performed 
by VoiceTone.sup.SM or a process associated with VoiceTone.sup.SM inasmuch as the 
user is on a telephone call (Bangalore [0030]). 

Further, Bangalore teaches [0057] FIG. 2C illustrates the response from the 
website when a user inputs a starting address 1600 Pennsylvania Avenue (the White 
House) to 50 Massachusetts Ave (Union Station, Washington DC). Information includes 
the total distance 1 .62 miles, and total estimates time: 6 minutes. Detailed directions 
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are provided 242 to the user and several options enable the user to receive the fastest 
route 252, the shortest route 244 or a route that avoids highways 246. The user can 
receive the directions via email 254 or the directions can be sent to a PCS phone 248. 
The form parser will analyze either off-line or dynamically each of the webpages 
illustrates in 2A, 2B and 2C to generate the appropriate prompts and input tags to 
receive via a voice dialog the information. As an example, after receiving via the voice 
dialog the "from" and "to" addresses, the service will receive the directions webpage 
shown in FIG. 2C. The form parser 106 will then identify the various pieces of 
information on the webpage 250 and dynamically adjust the dialog to ask the user 
questions such as: "Would you like to receive the directions via email or by phone?" If 
the response is by email, then the service receives the email address from the user or 
from another database or storage location and completes the process. The service may 
provide the directions (steps 1-5 on the webpage 250) via the voice dialog (Bangalore 
[0057-0058]). 

Additionally, Bangalore implements communication between a human and a 
machine to fill fields of a form, wherein Bangalore teaches the improvement of user 
information by implementing various prompts rather than assuming a user does not 
need any additional information ([0038-0052], particularly [0045-0050] diverging a bit 
from the topic of conversation to assist the user). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak to incorporate 
obtaining a value for at least one field of a form, receiving a user response that includes 
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an answer to the first prompt indicative of a value for the first field of the form, wherein 
the user response includes additional information that is not an answer to the first 
prompt and is associated with one or more of the semantic items, providing an 
additional prompt for a question control associated with a second field of the form based 
on the one or more semantic items associated with the additional information, wherein 
providing the additional prompt departs from the selected order of the question controls, 
and after the user has provided an answer to the additional prompt, returning to the 
selected order of the question controls to provide a next prompt for a next question 
control in the selected order as taught by Bangalore to allow for dynamically adjusting 
the dialog between a human and a computer (Bangalore [0057-0058], and Fig. 2A, 
voice to text input into fields on a form), wherein rather than assuming a user does not 
need any additional information, the system diverges from the topic of conversation to 
assist the user ([0038-0052], particularly [0045-0050], inquiring about a phone number 
to a location when a user did not inquire about a phone number to further assist before 
returning back to "is there anything else I can do for you today?"). 

Re claim 19, Alpdemir teaches a computer-implemented system comprising: 
a set of controls used on a server remote from a client for defining a dialog, the 
set of controls configured to generate client side markup in accordance with the dialog 
([0139]), wherein the set of controls comprises a plurality of question controls for 
generating markup related to audible prompting of a plurality of questions ([0226]) and 
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at least one answer control for generating markup related to a grammar for recognition 
([0169]), wherein a selected order for the set of controls is defined by an author to 
prioritize prompting of the plurality of questions ([0337-0339]); 

a server module, which when executed on the server, generates client side 
markup in accordance with the dialog; 

a client module executed on a processor of a computing device associated with 
the client using the client side markup to create the dialog as a function of the set of 
controls and the selected order ([0217]), wherein the client module maintains a plurality 
of semantic items that are associated with the plurality of question controls and are 
maintained in an ordered list ([0217]), wherein the plurality of semantic items are 
configured to store information related to responses received from the user for the 
question controls ([0250-0338] & Fig. 5, examples illustrating prompt and response in a 
dialog environment, that can start a users dialog over again); 

wherein the client module follows the selected order until a response is received 
from a user that includes both an answer to a prompt of a question that was given 
([0217]) and additional information provided by the user in the response with the answer 
not being an answer to a prompt that was given ([0252]-[0287], various answers and 
questions dependent on previous answers and questions), wherein the client module 
stores the additional information in one or more semantic items in the ordered list 
([0250-0338] & Fig. 5, examples illustrating prompt and response in a dialog 
environment, that can start a users dialog over again) such that the one or more 
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semantic items for the additional information are located in the ordered list before a 
semantic item for next question control for a next prompt in the ordered list ([0217]); 

wherein the client module subsequently accesses the ordered list to depart from 
the selected order by providing an additional prompt for the additional information to 
obtain ([0252]-[0287], various answers and questions dependent on previous answers 
and questions), from the user, a value for a field of the form before returning to the 
selected order ([0250-0338] & Fig. 5, examples illustrating prompt and response in a 
dialog environment, that can start a users dialog over again). 

However, Alpdemir fails to teach a client side markup for a client in a client/server 
system 

Albayrak teaches an interactive voice response system includes a server and a 
set of mobile clients. The server and clients include RF transceivers for exchanging 
messages over an RF channel. Each mobile client includes a microphone, a speaker or 
headset, a processor and a voice browser. The voice browser interprets voice pages 
received from the server. Upon receiving a particular voice page from the server, the 
voice browser outputs via the speaker voice prompts specified by the voice page. A 
speech recognition engine used by the voice browser converts voice responses from a 
user into a text response. The voice browser then performs an action based on the text 
response. The action taken may be to request a new voice page from the server, or to 
continue to interpret the current voice page. The server preferably includes an HTTP 
server module for receiving and responding to requests for voice pages from the mobile 
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clients in accordance with a predefined protocol. The mobile clients each include a text- 
to-speech module for converting text in a voice page into voice prompts, and a digitized 
speech module for playing digitized voice data representing other voice prompts. The 
mobile clients also include a speech recognition module for recognizing words or data 
string within a user's voice responses in accordance with a user specific voice file 
received from the server (Albayrak Col. 3 lines 3-27). 

Further, Albayrak teaches that Hypertext refers to a collection of computer- 
readable text documents containing links, that is, location references. A browser utilizes 
the links to facilitate moving its attention between linked documents. A voice browser is 
similar to a graphical browser in that it is a program that processes hypertext and 
presents the hypertext content in a specified format. The voice browser used in the 
preferred embodiment of this invention receives and outputs all information in the form 
of sound rather than having graphical input and output. The particular type of hypertext 
used in the preferred embodiment is based on VoiceXML. VoiceXML was designed by 
the VoiceXML Forum to create audio dialogs that feature digitized audio and speech 
recognition and to facilitate web-based development and content delivery. (Albayrak 
Col. 4 lines 11-39). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir to incorporate a client side 
markup for a client in a client/server system as taught by Albayrak to allow for a server- 
client voice browsing system, wherein markup language is utilized to convert text to 



Application/Control Number: 10/755,623 Page 29 

Art Unit: 2626 

speech, particularly for a mobile client that can prompt a user wirelessly (Albayrak Col. 
3 lines 3-27). 

However, Alpdemir in view of Albayrak fails to teach obtaining a value for at least 
one field of a form 

Bangalore teaches for example On Mapquest.RTM., to get driving directions, a 
user would click on the driving directions button 214. Other buttons include a maps 
button 212 and a road trip planner 216. FIG. 2B illustrates the forms to fill out for driving 
directions. On this webpage 230, there is a starting address 218 and a destination 
address 220. The information is filled into field 222 for a starting address and the field 
224 for a destination address. The present invention involves generating the necessary 
information drawn from the VoiceTone.sup.SM dialog to submit a request from the 
VoiceTone.sup.SM application to populate the necessary fields, for example in 
Mapquest, to obtain information. The interaction with the webpage 230 is all performed 
by VoiceTone.sup.SM or a process associated with VoiceTone.sup.SM inasmuch as the 
user is on a telephone call (Bangalore [0030]). 

Further, Bangalore teaches [0057] FIG. 2C illustrates the response from the 
website when a user inputs a starting address 1600 Pennsylvania Avenue (the White 
House) to 50 Massachusetts Ave (Union Station, Washington DC). Information includes 
the total distance 1 .62 miles, and total estimates time: 6 minutes. Detailed directions 
are provided 242 to the user and several options enable the user to receive the fastest 
route 252, the shortest route 244 or a route that avoids highways 246. The user can 
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receive the directions via email 254 or the directions can be sent to a PCS phone 248. 
The form parser will analyze either off-line or dynamically each of the webpages 
illustrates in 2A, 2B and 2C to generate the appropriate prompts and input tags to 
receive via a voice dialog the information. As an example, after receiving via the voice 
dialog the "from" and "to" addresses, the service will receive the directions webpage 
shown in FIG. 2C. The form parser 106 will then identify the various pieces of 
information on the webpage 250 and dynamically adjust the dialog to ask the user 
questions such as: "Would you like to receive the directions via email or by phone?" If 
the response is by email, then the service receives the email address from the user or 
from another database or storage location and completes the process. The service may 
provide the directions (steps 1-5 on the webpage 250) via the voice dialog (Bangalore 
[0057-0058]). 

Additionally, Bangalore implements communication between a human and a 
machine to fill fields of a form, wherein Bangalore teaches the improvement of user 
information by implementing various prompts rather than assuming a user does not 
need any additional information ([0038-0052], particularly [0045-0050] diverging a bit 
from the topic of conversation to assist the user). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak to incorporate 
obtaining a value for at least one field of a form as taught by Bangalore to allow for 
dynamically adjusting the dialog between a human and a computer (Bangalore [0057- 
0058]), wherein rather than assuming a user does not need any additional information, 
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the system diverges from the topic of conversation to assist the user ([0038-0052], 
particularly [0045-0050], inquiring about a phone number to a location when a user did 
not inquire about a phone number to further assist before returning back to "is there 
anything else I can do for you today?"). 



5. Claims 9, 10, 15, 17, 18, and 20-22 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Alpdemir US 20020035474 A1 (hereinafter Alpdemir) in 
view of Albayrak et al. US 6662163 B1 (hereinafter Albayrak) and Bangalore et al. 
US 20050135571 A1 (hereinafter Bangalore) and further in view of Takebayashi et 
al.3 US 5357596 A (hereinafter Takebayashi). 

Re claims 9 and 21 , Alpdemir teaches the computer readable storage medium of 
claim 8 wherein module maintains the information related to an order of responses 
([0337-0339]) received from the user as a stack. 

However, Alpdemir in view of Albayrak and Bangalore fails to teach an order of 
responses received from the user as a stack 

Takebayashi teaches an order table shown in FIG. 7 indicates the content of the 
order made by the input speech as understood by the system at each moment during 
the order taking operation, in a form of an order list similar to the ORDER TABLE frame 
of the semantic response representation, and this order table is to be updated according 
to the ACT frame and the ORDER TABLE frame of the semantic utterance 
representation supplied from the speech understanding unit 1 1 . On the other hand, the 
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past order table shown in FIG. 8 indicates the order table at a time of an output of the 
previous system response, i.e., the content of the order taken up to an output of the 
previous system response. This past order table of FIG. 8 is utilized as the dialogue 
history indicative of the change of the order table in the course of the order taking 
operation (Takebayashi Col. 11 lines 33-51 & Fig. 7-8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak and 
Bangalore to incorporate an order of responses received from the user as a stack as 
taught by Takebayashi to allow for the logging of data history, wherein the order and 
previous responses are stored for the dialog manager, wherein changes in user 
responses can be monitored for an improved recognition scheme (Takebayashi Col. 11 
lines 33-51 & Fig. 7-8, wherein ordered data in memory is construed to be functionally 
equivalent to a stack). 

Re claims 10 and 20, Alpdemir in view of Albayrak and Bangalore fails to teach 
the computer readable storage medium of claim 9 wherein the stack is of selected 
length such that the oldest information related to the oldest received response is 
removed when information is received related to the latest response from the user. 

Takebayashi teaches an order table shown in FIG. 7 indicates the content of the 
order made by the input speech as understood by the system at each moment during 
the order taking operation, in a form of an order list similar to the ORDER TABLE frame 
of the semantic response representation, and this order table is to be updated according 
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to the ACT frame and the ORDER TABLE frame of the semantic utterance 
representation supplied from the speech understanding unit 1 1 . On the other hand, the 
past order table shown in FIG. 8 indicates the order table at a time of an output of the 
previous system response, i.e., the content of the order taken up to an output of the 
previous system response. This past order table of FIG. 8 is utilized as the dialogue 
history indicative of the change of the order table in the course of the order taking 
operation (Takebayashi Col. 11 lines 33-51 & Fig. 7-8). 

Further, Takebayashi teaches the confirmation for the partial change of the order 
such as addition, replacement, and deletion is carried out by using only the speech 
response and the text data of the speech response. However, the visual information 
may also be used for the confirmation of the partial change of the order. In such a case, 
the display of the content visualizing image indicating the entire order may be 
interrupted temporarily, if desired. (Takebayashi Col. 27 lines 8-16). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak and 
Bangalore to incorporate a stack that is of selected length such that the oldest 
information related to the oldest received response is removed when information is 
received related to the latest response from the user as taught by Takebayashi to allow 
for the logging of data history, wherein the order and previous responses are stored for 
the dialog manager and older data is removed as the most recent response is received, 
wherein changes in user responses can be monitored for an improved recognition 
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scheme (Takebayashi Col. 1 1 lines 33-51 & Fig. 7-8, wherein ordered data in memory is 
construed to be functionally equivalent to a stack). 

Re claim 15, Alpdemir teaches the computer implemented method of claim 14 
wherein the set of controls includes an attribute to indicate whether a response to a 
prompt will be maintained in the information related to the order of responses received 
from the user ([0337-0339]), and wherein creating the dialog includes maintaining 
information related to an order of responses received from the user as a function of the 
corresponding attribute for a prompt ([0226]). 

Re claim 18, computer implemented method of claim 14, wherein defining a 
dialog includes logic for modifying the maintained information related to an order of 
responses received from the user([0337-0339]), and wherein creating the dialog 
includes modifying the maintained information pursuant to the logic ([0213). 

Re claim 1 7, Alpdemir in view of Albayrak and Bangalore fails to teach the 
computer implemented method of claim 16, wherein maintaining the ordered list 
comprises maintaining the ordered list in a stack 

Takebayashi teaches an order table shown in FIG. 7 indicates the content of the 
order made by the input speech as understood by the system at each moment during 
the order taking operation, in a form of an order list similar to the ORDER TABLE frame 
of the semantic response representation, and this order table is to be updated according 
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to the ACT frame and the ORDER TABLE frame of the semantic utterance 
representation supplied from the speech understanding unit 1 1 . On the other hand, the 
past order table shown in FIG. 8 indicates the order table at a time of an output of the 
previous system response, i.e., the content of the order taken up to an output of the 
previous system response. This past order table of FIG. 8 is utilized as the dialogue 
history indicative of the change of the order table in the course of the order taking 
operation (Takebayashi Col. 11 lines 33-51 & Fig. 7-8). 

Further, Takebayashi teaches the confirmation for the partial change of the order 
such as addition, replacement, and deletion is carried out by using only the speech 
response and the text data of the speech response. However, the visual information 
may also be used for the confirmation of the partial change of the order. In such a case, 
the display of the content visualizing image indicating the entire order may be 
interrupted temporarily, if desired. (Takebayashi Col. 27 lines 8-16). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Alpdemir in view of Albayrak and 
Bangalore to incorporate maintaining the ordered list comprises maintaining the ordered 
list in a stack as taught by Takebayashi to allow for the logging of data history, wherein 
the order and previous responses are stored for the dialog manager and older data is 
removed as the most recent response is received, wherein changes in user responses 
can be monitored for an improved recognition scheme (Takebayashi Col. 11 lines 33-51 
& Fig. 7-8, wherein ordered data in memory is construed to be functionally equivalent to 
a stack). 
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Re claim 22, Alpdemir teaches the system of claim 20 wherein the ordered list is 
indicative of a list of semantic items ([0250-0338] & Fig. 5, examples illustrating prompt 
and response in a dialog environment, where the best matching data to a user response 
is selected from a set of data). 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael C. Colucci whose telephone number is (571)- 
270-1847. The examiner can normally be reached on 9:30 am - 6:00 pm, Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571)-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 



Application/Control Number: 10/755,623 Page 37 

Art Unit: 2626 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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